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ABSTRACT 

Background Patients are safer and receive 
higher quality care when providers work as a 
highly effective team. Investment in optimising 
healthcare teamwork has swelled in the last 
10 years. Consequently, evidence regarding the 
effectiveness for these interventions has also 
grown rapidly. We provide an updated review 
concerning the current state of team-training 
science and practice in acute care settings. 
Methods A PubMed search for review articles 
examining team-training interventions in acute 
care settings published between 2000 and 2012 
was conducted. Following identification of 
relevant reviews with searches terminating in 
2008 and 2010, PubMed and PSNet were 
searched for additional primary studies published 
in 201 1 and 2012. Primary outcomes included 
patient outcomes and quality indices. Secondary 
outcomes included teamwork behaviours, 
knowledge and attitudes. 
Results Both simulation and classroom-based 
team-training interventions can improve 
teamwork processes (eg, communication, 
coordination and cooperation), and 
implementation has been associated with 
improvements in patient safety outcomes. 
Thirteen studies published between 201 1 and 
2012 reported statistically significant changes in 
teamwork behaviours, processes or emergent 
states and 10 reported significant improvement 
in clinical care processes or patient outcomes, 
including mortality and morbidity. Effects were 
reported across a range of clinical contexts. 
Larger effect sizes were reported for bundled 
team-training interventions that included tools 
and organisational changes to support 
sustainment and transfer of teamwork 
competencies into daily practice. 
Conclusions Overall, moderate-to-high-quality 
evidence suggests team-training can positively 
impact healthcare team processes and patient 
outcomes. Additionally, toolkits are available to 
support intervention development and 
implementation. Evidence suggests bundled 
team-training interventions and implementation 
strategies that embed effective teamwork as a 



foundation for other improvement efforts may 
offer greatest impact on patient outcomes. 



INTRODUCTION 

...It has become necessary to develop 
medicine as a cooperative science; the 
clinician, the specialist, and the labora- 
tory workers uniting for the good of the 
patient, each assisting in elucidation of 
the problem at hand, and each depend- 
ent upon the other for support. - 
William J. Mayo, Commencement 
speech at Rush Medical College, 1910 

Deficiencies in communication and team- 
work have long been cited as a frequent 
contributor to adverse events. Precise esti- 
mates of the extent of the problem are dif- 
ficult to make, given definitional, 
reporting and measurement inconsisten- 
cies. However, a variety of studies support 
the notion that teamwork and communi- 
cation are critical components of safe 
healthcare systems. Previous reviews 
report linkages between various aspects of 
teamwork (eg, situational monitoring, 
communication, leadership, trust, shared 
mental models) and clinical perform- 
ance. 1-3 Meta-analytic results suggest the 
relationship between team processes and 
clinical performance indicators has gener- 
ally been characterised by medium to 
large effect sizes. 4 5 For example, studies 
in surgery have shown increased odds of 
complications and death (OR 4.82; 95% 
CI 1.30 to 17.87) when surgical teams 
exhibit less frequent teamwork behaviours 
(eg, less information sharing during 
intraoperative and handoff phases, and 
less briefing). 6 Reviews of malpractice 
claims further underscore that communi- 
cation problems are major contributing 
factors in 24% of cases. 7 Other studies 
found teamwork and communication 
issues cited as root causes in 52-70% of 
adverse events. 8 9 Additionally, teamwork 
and communication dimensions of safety 
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culture have been significantly related to adverse clin- 
ical events. 10 11 

The 2001 Making Health Care Safer report 12 pro- 
vided one of the early reviews concerning the topic of 
healthcare team-training in a chapter entitled 'Crew 
Resource Management and Its Application in 
Medicine'. This review discussed early conceptualisa- 
tions of team-training in other high-reliability indus- 
tries such as aviation and summarised early studies 
attempting to translate team-training principles devel- 
oped elsewhere into healthcare settings. The develop- 
ment and implementation of team-training 
programmes in acute care settings has grown dramat- 
ically in the last decade with improvements in 
content, methods and evaluation desgins. 13-15 
Advances in training content, implementation and 
evaluation have increasingly drawn on over 30 years 
of evidence examining team performance processes 
and team-training across a wide variety of high-risk 
environments. 16 While previous reviews described the 
state of team-training in healthcare through 
2009, 13 14 the prevalence of peer-reviewed literature 
evaluating team-training interventions continues to 
grow. Publication trends suggest that in the last 
5 years, over 448 papers examining teamwork topics 
in a variety of languages, across a variety of care set- 
tings, have appeared in PubMed (see figure 1). 
Additionally, team-training was formally identified in 
2013 as a top patient safety strategy encouraged for 
adoption in acute care settings. 17 Therefore, our 
objective is to provide an updated narrative synthesis 
of the body of evidence evaluating team-training in 
acute care settings, including implementation trends, 
evidence for effectiveness and potential boundary 
conditions (ie, factors that may enhance or inhibit 
effectiveness). In providing this updated review, we 
hope to also highlight fruitful areas for future research 
and innovative practice. 

Types of team-training strategies 

Team-training is defined as a constellation of content 
(ie, the specific knowledge, skills and attitudes that 
underlie targeted teamwork competencies), tools (ie, 
team task analysis, performance measures) and deliv- 
ery methods (ie, information, demonstration and 
practice-based learning methods) that together form 
an instructional strategy. 18 In this sense, team-training 
is a systematic methodology for optimising the com- 
munication, coordination and collaboration of health- 
care teams that combines specific content with 
opportunities for practice, formative feedback and 
tools to support transfer of training to the daily care 
environment. 
As described by the National Quality Forum 19 : 

Health care organizations must establish a proactive, 
systematic, organization-wide approach to developing 
team-based care through teamwork training, skill 



building, and team-led performance improvement 
interventions that reduce preventable harm to 
patients... training programs should systematically 
address and apply the principles of effective team lead- 
ership, team formation [and team processes]. 

Borrowing from other high-reliability communities, 
team-training in healthcare originated largely in the 
form of Crew Resource Management (CRM), a spe- 
cific team-training strategy focused on developing a 
subset of teamwork competencies including hazard 
identification, assertive communication and collective 
management of available resources. 20-23 However, the 
practice of team-training is much broader. Today, 
team-training is an overarching term that encompasses 
a broad range of learning and development strategies, 
methods and teamwork competencies. Table 1 out- 
lines several types of team-training strategies. 

The critical element defining team-training is that 
the learning activity focuses on developing, refining 
and reinforcing knowledge, skills or attitudes that 
underlie effective teamwork behaviours such as com- 
munication, coordination and collaboration. Prior 
reviews found that the most commonly targeted team- 
work competencies include communication, situ- 
ational awareness, leadership, role clarity and 
coordination. 14 24-27 To this end team-training 
activities are often designed to develop generalisable, 
transportable teamwork competencies that learners 
can apply across different settings and teams. This 
differentiates team-training from learning activities 
focused on technical clinical skills (eg, differential 
diagnosis and procedural skills), as well as team 
building exercises focused on developing emergent 
states such as trust or cohesion among members of an 
intact team. 

METHODS 

A PubMed search for key review articles examining 
team-training interventions in acute care settings pub- 
lished between January 2000 and December 2012 
was conducted. Key search terms included 'team- 
training', 'teamwork training', 'teamwork training 
interventions', 'crew resource management', 
'TeamSTEPPS' and related terms. Following identifica- 
tion of relevant reviews with searches terminating in 
2008, 13 2009 12 21 and 20 1 0, 24 25 a search of PubMed 
and PSNet was completed to identify additional 
primary studies published between 1 January 2011 
and 31 December 2012. Studies were excluded if they 
were only descriptive in nature, if conducted in 
non-English-speaking populations or if primarily tar- 
geting students or trainees. Narrative summaries of 
individual studies and shared themes are presented. 
This review was supported by Agency for Healthcare 
Research and Quality (AHRQ), which had no role in 
the selection or review of the evidence or the decision 
to submit this manuscript for publication. 



360 



Weaver SJ, ef al. BMJ Qual Saf 2014;23:359-372. doi: 1 0.1 1 36/bmjqs-201 3-001848 



Narrative review 




RESULTS 

Several previous reviews have examined the effective- 
ness and implementation of healthcare team- 
training. 2 13-15 23 ~ 33 p or example, one systematic 
review of interventions to improve team effectiveness 
in healthcare found that the majority involved some 
form of team-training (42 of 48 reviewed studies). 13 
Other reviews investigated the effectiveness of team- 
training for obstetric emergencies, 33 for enhancing 
communication in surgery and anaesthesia, 20 25 31 
classroom-based interventions 27 and simulation-based 
interventions. 15 24 30 Additionally, several reviews 
investigated the content, design and delivery of team- 
training and the impact of team processes on clinical 
processes and outcomes. 2 14 24 26 29 31 32 34 We draw 
on results from these reviews and 26 articles published 
since their searches terminated in order to describe the 
current body of evidence regarding the effectiveness of 
team-training in acute care settings, the contexts in 
which it has been evaluated and effects of different 
design and delivery methods. Findings from 26 studies 
published after previous reviews concluded their 
searches are summarised in table 2 and discussed in 
greater detail in subsequent sections (see online supple- 
mentary appendix for detailed evidence summary). 

What is the context for team-training? 

Previous reviews highlight that team-training has been 
implemented across a broad range of acute care con- 
texts. 2 13 14 27 30 This includes academic hospitals 35 
and community-based hospitals, 36 37 as well as medical 
centres affiliated with the Veterans Administration (VA) 
and the Military Health System. 38 ^* 0 Though our 
focus is on current practitioners, team-training 
programmes have focused on a variety of audiences 
including students and clinical trainees. 45 46 The 
majority of studies have focused on evaluating team- 
training efforts among frontline clinicians working in 
obstetric or perioperative care areas; however, studies 



have also been conducted in emergency and critical 
care, procedural areas (eg, endoscopy) and administra- 
tion. For example, table 2 demonstrates that 9 studies 
of the 26 published between 2011 and 2012 were con- 
ducted in surgical work areas, 5 were conducted in 
critical care or inpatient nursing units and 4 were con- 
ducted in obstetrical or emergency/trauma settings, 
respectively. Four studies were conducted in other 
areas, including procedural areas or administration, or 
were conducted across multiple types of care areas. 

Team-training programmes for hospital leaders have 
also demonstrated positive effects. Individual execu- 
tive leadership development programmes fall outside 
the scope of traditional team-training initiatives; 
however, team-training programmes for executive 
leaders is an area ripe for further research as many 
healthcare organisations move towards enterprise 
affiliations and Accountable Care Organization (ACO) 
models. In this way administrators, service line 
leaders, division leaders and frontline providers 
increasingly manage and work not only in teams, but 
as part of teams-of-teams, or multiteam systems 
(MTS). 47 48 MTSs are defined as two or more teams 
that work interdependently towards at least one 
common goal. 49 Initial work has begun to examine 
the leadership competencies that matter most in these 
contexts and to test interventions to cultivate 
them. 50 51 However, further development and evalu- 
ation of team-training interventions targeting MTSs 
within the healthcare context are needed. 

Team type 

Teamwork in healthcare is often characterised by 
highly dynamic team membership, participation on 
multiple teams and rapid team formation. As opposed 
to intact teams who have a history and future working 
together, some types of healthcare teams, particularly 
in acute care settings, may not have ever worked 
together before, form under stressful conditions (eg, a 
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Table 1 Team-training strategies 



Team-training 
strategy 



Definition 



Primary teamwork 
competencies targeted 



Best practices 



Assertiveness training 



Cross -training 



Error management 
training 



Guided team 
self-correction 



Dedicated to developing communication 
strategies that support task-relevant and 
team-performance relevant assertiveness 



Team members learn the roles that comprise 
the team, as well as the tasks, duties, and 
responsibilities fulfilled by fellow team 
members. 



Participants are encouraged to make errors 
during training scenarios, analyse these errors 
and practice error recognition and 
management skills. 



Strategy designed around a cycle of facilitated 
briefings and debriefings that occur around a 
training scenario or live event. 



Backup behaviour 
Closed-loop 
communication 
Conflict management 
Mutual trust 
Psychological safety 
Team leadership 



Accurate and shared 
mental models (SMMs) of 
team roles and 
responsibilities 



Collective efficacy 
Cue-strategy associations 
SMMs 

Team adaptation 



Backup behaviour 
Collective orientation 
Closed-loop 
communication 
Cue-strategy associations 
Mission analysis 
Mutual trust 
SMMs 

Team adaptation 
Team leadership 



Clearly define training objectives around 
task-relevant and team performance 
assertiveness rather than general 
assertive behaviours and differentiate 
from aggressive behaviours. 
Compare and contrast effective and 
ineffective assertive behaviours 
Provide opportunities to practice 
appropriate assertiveness that include 
feedback. Practice should also strive to 
include realistic time pressures or other 
stressors to allow practice using and 
reacting to appropriate assertiveness 
under such conditions. 

Include information about the roles and 
responsibilities of other team members 
and how they operate to achieve these. 
Explain the why — clarify who members 
depend on for information. 
Provide opportunities to shadow another 
role if possible. 

Provide feedback during cross-training 
that facilitates the formation of 
reasonable expectations of one another. 

Ensure trainees understand the purpose 
of this training strategy is to encounter 
errors and to have the opportunity to 
practice managing them in a safe 
environment. 

Frame errors as positive opportunities for 
learning. 

Embed the opportunity to make errors 
into training scenarios by providing 
minimal guidance during the scenario. 
Follow the scenario with immediate 
feedback and discussion to facilitate 
learning. 

Define the team self-correction skills to 
be trained prior to team self-correction 
training. 

Record positive and negative examples 
of teamwork dimensions during team 
performance episode. 
Classify and prioritise observations, 
diagnose strengths and weaknesses, and 
identify goals for improvement before 
beginning debrief. 

Set the stage for team participation and 
solicit examples of teamwork behaviour 
during debrief. 

Continued 



362 



Weaver SJ, ef al. BMJ Qual Saf 2014;23:359-372. doi: 1 0.1 1 36/bmjqs-201 3-001848 



Narrative review 



Table 1 Continued 



Team-training 
strategy 



Definition 



Primary teamwork 
competencies targeted 



Best practices 



Metacognition 
training 



Team adaptation and 
coordination training 



Focuses on developing cognitive aspects of 
team performance by teaching strategies 
dedicated to analysing, updating and aligning 
mental models of the s task, coordination 
strategy, and contingencies. 

Focuses on how to effectively use all available 
resources (ie, people, information, etc.) through 
effective team communication, coordination 
and cooperation. Crew or Crisis Resource 
Management is a form of TACT. 



Cue-strategy associations 
Mission analysis 
SMMs 

Team adaptation 



► Backup behaviour 

► Closed-loop 
communication 

► Cue-strategy associations 

► Mission analysis 

► Mutual performance 
monitoring 

► Leadership 

► Shared mental models 



Develop training objectives around cognitive 
processes such as planning, monitoring and 
re-analysis. 

Structure metacognitive practice tasks around 
a subject that trainees have pre-existing 
knowledge about. 

► Develop training objectives that address 
around transportable teamwork 
competencies for ad-hoc teams (no 
history or future). 

► Training team-specific competencies can 
also be incorporated for intact teams. 

► Train intact teams together if possible. 

► Create opportunities for both guided 
and unguided practice. 

► Develop feedback mechanisms that 
engage self-reflection and team 
self-correction following practice 
opportunities. 

► Develop tools that support effective 
teamwork, but recognise that tools 
alone (eg, checklists) cannot optimise 
team performance (and alone may 
negatively impact performance). 



Adapted from Salas ef ai: 



code) and may not have great likelihood of working 
together closely in the future. Such teams are known 
as ad hoc teams or action teams and are defined by 
rapid formation, an abbreviated lifespan and often 
limited experience working together previously. Given 
this context, can team-training be effective for such ad 
hoc teams? Previous reviews indicate that healthcare 
team-training has been conducted with intact (ie, 
teams who work together currently or on a regular 
basis) and ad hoc teams (ie, teams formed quickly for 
brief periods of performance or for training purposes 
only). For example, Weaver 14 found eight studies that 
reported training intact teams and five studies that 
reported training in ad hoc teams. One meta-analysis 
that examined team-training effectiveness across a 
variety of industries including healthcare found 
similar effect sizes for teams who worked together on 
a regular basis (intact teams p=0.48) and teams who 
did not (ad hoc teams p=0.44). 4 A second 
meta-analysis that also examined the effects of team- 
training on overall team performance across a range 
of settings found that team-training had a greater 
positive impact on the performance of ad hoc teams 
(d Intact =0.62; dAdhoc = 0.92); however, the difference 
was not statistically significant. 52 Within healthcare, 
several of the most robust studies have used training 
strategies allowing teams to train together (eg, closing 
the operating room for 1 day so that team members 



could attend training together); however, this is not a 
boundary condition for training effectiveness. 

What have we learned about team-training design and 
delivery? 

Several reviews of team-training and team processes in 
healthcare have also examined training design and 
delivery. 13 14 24 27 31 These reviews find variation 
among programmes in the teamwork competencies 
targeted, curriculum used, how much time learners 
spend in training, how often clinicians and staff are 
participating, and other details regarding content, 
delivery strategies and evaluation efforts. 

Training content 

As described earlier, many different types of training 
strategies fall under the umbrella of team-training. 
Previous reviews found communication, situational 
awareness, leadership and situation monitoring as the 
most common teamwork competencies targeted. 14 27 
Table 2 emphasises that similar competencies have 
also been the focus of more recent primary studies, 
particularly as the adoption of teamwork training cur- 
riculums built around these competencies (eg, 
TeamSTEPPS, CRM, VA Medical Team Training (VA 
MTT)) has grown. For example, 9 of the 26 studies 
reviewed used some form of CRM, 7 reported using 
components of the TeamSTEPPS curriculum and 3 
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Table 2 Studies examining team-training effects on clinical process or patient outcomes 
Study characteristics 



Curriculum Setting Design Clinical processes or patient outcomes 



Andreatta 2011 41 


OBEMAN 


OB 


Descriptive 


► 


rive LyptrS 01 IIILOliyiUcIlL ilObpildl pOIILIcS Ul piOLcUUicS 












nmiQcninn rliniru nr^rtiro inontiTiori 
yUVcllllliy LIIIIILdl pidLULc lUcllUllcU 


A ir ^ Pi 1 1 42 

Armour zu 1 1 


TC 
I J 


AD 

UK 


Time series 


► 


Significant reduction in surgical morbidity (20.2% vs 11% 












*) and mortality (2.7% vs 1%*) 










► 


Significant improvement in 4 of 6 SQIP measures* 










► 


Room turnover time decreased significantly (43 vs 35.5 












min ), percentage of on-time first case starts improved 












(69% to 81%+) 










► 


n i ■ ■ ' 1 1 ' i 1 ■ ■ f' i 1 ■ 1 

Patient willingness to recommend significantly improved 












(77% to 89.3%*) 










► 


Evidence of some decay over 1 year post-training 












£ il t 4. 1*4. L'J'4. 4.' 4. 4. J 

follow-up for mortality, morbidity, on-time starts and 












patient satisfaction, while SQIP remained improved 


Carney 201 1 


VA MTT 


OR 


Pre-post 






Castner 201 2 s5 


TS 


Multiple 


NCGPT 






Cooper 20 11 53 


Leadership TT 


Hospital 


Post only 










management 








hoorinn 9P.1 1^ 
Uccl II ly ZU I I 


i _) 


^Ull lUdL SUppUl L 


Pro r\f\c+ 

ric-puM 


► 


Decreased number of adverse events (22.2 vs 18.2 






hospital 






events + ) 










► 


P. 3 OA rlorro^cQ in murlir^tinn ^nrl tr^ncfi icir\n onrirc \~l 1 wc 
Oj /0 UeUedbc III 1 1 IcUILdUUI I dllU. LldllblUblUII cllUlb \/. I vb 












1 7*\ 70% Hprrp^<;p in nppHlp^tirlc ininrv 3nH pynn^nrp^ 

I . L. ff 1 \J /U UCLI CGjC 1 J 1 1 ICCJICjUV_I\ II IJUI y U 1 IU CAUUjU 1 Cj 












(4.0 vs 1.2*), 65% decrease communication incidents 












(5.2 vs 1.8*) 










► 


No significant decrease in incidents related to three other 












teamwork competencies 


Figueroa 201 2 65 


TS 


ICU 


Time series 






Fransen 201 2 43 


Multidisciplinary 


OB 


Cluster RCT 




Trained teams adhered to predefined obstetric procedures 












more frequently than non-trained teams (83% vs 46%*) 


Frengley 2011 86 


CRM 


ICU 


NCGPPT 




Regardless of modality that clinical management skills 












were also taught, all participating teams improved clinical 












management scores* 


Heard 201 1 87 


CRM 


Endoscopy 


Time series 






k'ir^rhhai im Id 1 1^ 
Ml SLI [UdUM 1 £U 1 Z 


K/li itirlicnninarw 
IVIUILIUIbLipill Idly 


OR 


Pro-nnct 
rlc jJUM 






Maxson 201 1 89 


TS 


Surgery 


Time series 






Mayer 201 1 35 


TS 


PICU 


Longitudinal with 




Mficnrnmi^l infprtinnc rlprrp^cprl clinhtlw^ 
INLoULLM 1 Hal II 1 1 CLLIUI 1 b UcLlcabcU illUillly 






SICU 


non-equivalent control 




Av/pr^riP timp fnr nh^rinn n^tipntc nn PYtr^rnrnnrp^ 
MVclaytr Lllllc IUI piaLlliy pdLlcllLi UN cALIaLUI puical 












mpmhnnp nyunpnatinn (\-C\\/\0\ HprrP3c:pH "lA 

IllCIIIUIallC UAy UCI laLIUII ^LvJVIWy UCLI CCJjCU Vj 1 4 

min ) 












Nn cJnnifir^nt rh^nnp in Ipnnth nf raniH rp<;nnn<;p tpam 

IMU Jly 1 1 1 1 1 LU 1 1 I LIIQIIUC III ICIIULII Ul IUUIU ICjUUNjC ICQ 1 1 1 












events 


Mrl annhlin 301 1 44 

IVILLdliyi Mill ZU 1 1 


Tr3i i m zi TT 
1 IdUl I Id 1 1 


1 rauma 


Pnct-nn Iw 
rUM Ul ny 






Neily 20 1 1 38 


VA MTT 


OR 


RCC 


► 


JU /0 yicdLcl IcUULUOll 111 1 !SK-dUjUSLcU SUiyiLdl 111 Ol Lei 1 1 Ly 












for MTT group versus control (RR=1 .49)* 










► 


Reduction of 0.5 deaths per 1000 procedures associated 












with every quarter that teamwork intervention was in 












place* 










► 


Qualitative interviews suggested improvements in overall 












perioperative efficiency, reduced length of procedures, 












improved first case on-time start times and equipment use 



Continued 
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Table 2 Continued 



Study characteristics 





Curriculum 


Setting 


Design 


Clinical processes or patient outcomes 


Patterson 2013 (online 
20 12) 66 


Multidisciplinary 
TT 


Paediatric ED 


Time series 


► 


Patient safety event rate decreased (2-3 annually vs 
1000 days since safety event, 12 m post) 


Phipps 201 2 70 


CRM 


L&D 


Pre-post 


► 
► 


r"lorro:3co in ark/orco nutrnmo inHov iAOli rfl O^ - } nro \/c 
UcUcdbc III dUVclbc UULLUIIIc IIIUcA \r\\J\) \\J.\JDZ. pic vb 

D 043 nnstl 

U.UHJ UUjLJ 

No significant change in patient satisfaction 


Riley 2011 36 


TS 


L&D 


NCGPPT 


► 

w 


Weighted adverse outcome score (WAOS) — an index of 
perinatal harm — decreased 37% for full intervention 

nrm in nnlw M 1 R nrp O nn<;t*^ 
uiuuu uiiiy ^ i . i j uic vj \j . i l. uujL ) 

Variability in WAOS smrps was decreased nost-traininn fnr 

V CI 1 1 CI U 1 1 1 I y Ml V V r\\-/ -J jLUI Cj VVUJ UCLI CUJCU UUj I LI CI 1 1 1 1 1 \\J\ 

full intervention group only 


Singer 2011 54 


Leadership TT 


Hospital mgmt. 


Qual. Longitudinal 






Steinemann 201 1 90 


CRM 


Trauma 


Prospective cohort 


► 


16% reduction in ED resuscitation time*, 76% increase in 
completeness of clinical tasks* 
No significant changes in mean hospital length of stay, 
ICll davs or Heaths 

1 V_ ^-J U CI V -J CJ 1 CI CI 1 1 1 J 


Stevens 201 2 71 


CRM 


Cardiac surgery 


Pre-post 






Stacker 201 2 91 


CRM 


PICU 


Time series 






Tapson 2011 80 


CRM 


Surgery 


Longitudinal 


► 


Significantly more post-training than pretraining charts 
met guideline recommendations and standards of care for 
timing, inpatient duration and prophylaxis use beyond 
discharge* 


van Schaik 201 1 92 


CRM 


PICU 


Cross-sectional 






Volk 201 1 93 


CRM 


OR 


Post-only 






Young-Xu 20 11 58 


VA MTT 


OR 


RCC 


► 


20% greater reduction in risk-adjusted surgical morbidity 
in the MTT group versus control (RR=1 .20)* 



+, improvement, but not statistically significant; CBT, case based learning; CRM, Crew or Crisis Resource Management; L&D, labor and delivery; NCGPT, 
non-equivalent comparison group post-test only; NCGPPT, non-equivalent comparison groups pre-test/post-test; OB, obstetrics; OBEMAN, obstetrics, 
emergency medicine, anesthesiology, and neonatology program; OR, operating room; Peds, pediatrics; PICU, pediatric intensive care unit; RCC, retrospective 
controlled cohort study; SBT, simulation based training; SICU, surgical intensive care unit; TS, TeamSTEPPS; VA MTT, VA Medical Team Training. *p<0.05. 



studies used VA. MTT. Seven studies reported using 
other team-training curriculums. 

More narrowly defined programmes targeting deep 
development of specific teamwork skills (eg, leader- 
ship) in the healthcare context have also been evalu- 
ated. Two 53 54 of the twenty-six studies reviewed 
specifically evaluated an intervention designed to 
develop teamwork competencies among hospital 
leaders and administrators. Such programmes may 
differ from those focused on frontline providers in 
the level of emphasis on certain teamwork competen- 
cies (eg, transformational leadership, conflict manage- 
ment and mediation, broader internal and external 
environment awareness, boundary spanning activities 
that facilitate coordination with other internal or 
external teams, etc., may be of greater emphasis), 55 5 
as well as design (eg, greater use of table-top simula- 
tions or game-based approaches). 57 

Training modality and methods 

The evidence to date demonstrates that healthcare 
team-training programmes vary in the instructional 



modalities and methods used. Instructional methods 
can be conceptualised in terms of three broad categor- 
ies: (1) information-based methods (eg, didactic 
lecture), (2) demonstration-based methods (eg, behav- 
ioural modelling, videos) and (3) practice-based 
methods (eg, simulation, role playing). Previous 
reviews found that the majority (83%) of team- 
training programmes integrated both information and 
practice-based methods, while only 35% reported 
incorporated demonstration-based activities. 14 

Simulation-based versus classroom-based training. 
Simulation is a broad training modality that may 
include physical simulation of clinical care environ- 
ments, standardised patients, cognitive simulations 
and role play. It includes both centre-based learning 
activities and in situ activities that occur in the actual 
environment where care is provided. 

Simulation is a common modality for team-training 
in healthcare. Previous reviews found 68% of pro- 
grammes using simulation-based activities. 14 Several 
reviews have examined simulation-based team-training 
exclusively. 15 24 30 In terms of outcomes, previous 
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reviews examining simulation-based team- 
training 15 24 30 and classroom-based team-training 27 
separately reported similar positive effects on learner 
reactions, knowledge or skills, and clinical practices. In 
terms of patient outcomes, both simulation studies and 
classroom-based studies have shown a positive impact 
on patient outcomes. Though simulation is powerful 
learning activity, particularly for teamwork competen- 
cies, some of the most robust studies published since 
these previous reviews were completed did not use 
elaborate simulation methods, but still achieved signifi- 
cant reductions in both mortality and morbidity. 38 58 
Direct comparison studies also provide insight. Riley 36 
compared perinatal work areas in three different hospi- 
tals; one participated in a didactic team-training pro- 
gramme, one participated in an in situ simulation 
programme and one served as a control. Results indi- 
cated that significantly greater reductions in patient 
harm were achieved by the group that participated in 
the in situ simulation programme compared with both 
the didactic and control groups (37% reduction vs 1% 
reduction vs 43% increase in harm, respectively). 

Training duration. Programme duration is an area in 
which team-training programmes in healthcare vary. A 
review of 18 studies evaluating classroom-based team- 
training interventions found course duration varied 
from 4 h to 3 days with several studies describing 
longer train-the-trainer programmes. 27 Another 
review found that 53% of 40 reviewed team-training 
programmes were designed to last less than 1 day. 14 

Intervention design and implementation strategy 
Overall, no comprehensive meta-analysis to date has 
directly examined variations in design or implementa- 
tion strategy as boundary conditions influencing the 
effectiveness of healthcare team-training programmes. 
However, current evidence supports the notion that 
the most effective team-training programmes leverage 
a bundled intervention design that pairs learning 
activities and practice, with tools designed to support 
effective teamwork during daily clinical care (eg, 
checklists, reminders, peer coaches). This bundled 
design aligns with a systems-based approach to train- 
ing that emphasises a comprehensive strategy for 
developing and sustaining expert team performance. 32 
These strategies view training as one piece of the 
puzzle that will support effective team performance 
under stressful conditions over time. For example, 
Buljac-Samardzic's 13 descriptive systematic review of 
strategies to optimise team effectiveness found that 
the majority involved some form of team-training; 
however, other interventions also included tools to 
support team performance during care (eg, checklists, 
goal lists; 8 studies), as well as broader organisational 
interventions (eg, redesign of care processes or team 
structures, 8 studies). 

Implementation strategy. In terms of implementation 
strategy, both train-the-trainer and direct train-the-staff 



strategies have been used. For example, a 
train-the-trainer model formed the foundation for the 
National Implementation of TeamSTEPPS Project, 59 a 
collaborative effort by the U.S. Department of Defense 
(DoD), the AHRQ and the American Institute for 
Research (AIR) designed to create a national training 
and support infrastructure for healthcare entities 
implementing team-training. Through a national 
network of team resource centres, individuals inter- 
ested in leading the implementation of team-training 
within their organisation could become TeamSTEPPS 
Master Trainers by participating in an intensive 3 -day 
training session. Master Trainers then return to train 
administrators and frontline personnel within their 
own organisation using the customisable TeamSTEPPS 
curriculum. A slightly different approach was used in 
the large-scale implementation of team-training 
throughout the VA. As part of the ^A National Center 
for Patient Safety Medical Team-training (MTT) 
Programme, learning sessions for participating \A 
medical centres were facilitated directly by an interdis- 
ciplinary team of dedicated MTT faculty. 38 Both strat- 
egies, however, include local facility change teams, 
implementation of on-the-job tools (eg, process check- 
lists, scripts) to support training transfer, as well as 
measurement and evaluation processes as integral 
implementation components. 

Transfer of training. Overall, systems-based models 
of team-training share an implementation approach 
that strategically considers what happens before, 
during and after training in order to facilitate transfer 
of effective teamwork into daily care processes. These 
models are rooted in classic transfer of training 
models, which stipulate that transfer, generalisation of 
teamwork skills developed in training and mainten- 
ance of these skills over time is the product of the 
work environment (ie, support for using teamwork 
skills, opportunities to use these skills in practice), 
training design and individual trainee characteristics. 0 
This suggests that the most effective team-training 
efforts include readiness assessment and preparation, 
learning activities that bring together multidisciplinary 
practitioners, deployment of performance support 
tools (eg, checklists or structured communication 
tools) and post-training support through coaching, 
reinforcement and recognition for improved team- 
work, and ongoing measurement paired with con- 
structive feedback concerning teamwork processes. 61 
For example, the \A MTT programme was led by a 
national coordinating entity (the \A National Center 
for Patient Safety) who worked with local implemen- 
tation teams through 2 months of preplanning and 
readiness preparation prior to implementation. 38 
Transfer of training was also supported by structured 
communication tools and quarterly follow-up coach- 
ing. The TeamSTEPPS programme similarly offers 
readiness assessment and evaluation planning, and 
facilitator training as part of the TeamSTEPPS Master 
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Trainer curriculum, as well as coaching and deploy- 
ment support. 

Organisational policies and procedures have also 
been changed to facilitate transfer of trained team- 
work skills. For example, one study of team-training 
in surgery found significant improvements in on-time 
first case starts and operating room turnover times fol- 
lowing training. 42 To sustain these improvements, the 
organisation implemented policy changes and incen- 
tives around these two processes. The combination of 
team-training and these organisational changes 
resulted in continued improvement in both processes 
over the 4 years following training implementation. 
Specifically, on-time starts increased 42% across the 
total 5-year period reported (69-98%) and there was a 
38% decrease in OR turnover time (mean 2 oo6 = 43 min 
to mean 2 oio = 27 min). Such examples reinforce the 
notion that effective team-training initiatives go 
beyond training alone to proactively address aspects 
of the organisational work environment that support 
team members in sustaining effective teamwork prac- 
tices over time, as well as generalisation of skills to 
new teams and new situations. 2 

What have we learned about team-training effectiveness? 

Team-training has demonstrated the capacity to 
improve both how teams do their work and resulting 
outcomes across a wide variety of industries. For 
example, one meta-analysis of team-training that 
included 93 effect sizes across a broad range of indus- 
tries found that participation in team-training can 
account for nearly 20% of the variance in team pro- 
cesses (p=0.44) and outcomes (p=0.39). 

Several reviews also support the notion that team- 
training can meaningfully improve participant knowl- 
edge or attitudes, teamwork processes, clinical care 
processes and even patient outcomes. 5 13 14 27 31 
Though local customisation of team-training curricu- 
lum is common and evaluation metrics are heteroge- 
neous, we attempt to summarise findings in the 
remainder of this section in terms of participant reac- 
tions, changes in knowledge, attitudes or skill, team- 
work processes or behaviours in clinical 
environments, and clinical processes or patient out- 
comes. This organising structure is based upon 
Kirkpatrick's framework for training evaluation that 
organises training outcomes across the four levels 
described below. 63 

Level 1 : learner reactions to training 

Previous reviews demonstrate that learner reactions are 
one of the more common, yet least robust, evaluation 
metrics reported. One review found that 60% of 40 
reviewed articles reported learner reactions to training 
(eg, perceptions of training utility, applicability and 
general satisfaction with training). 14 The sample of 26 
studies summarised in table 2 demonstrates a shift 
towards more comprehensive evaluation of team- 



training programmes, however. Ten studies explicitly 
reported trainee reactions; while 7 studies examined 
trainee knowledge, attitudes or self-efficacy; 16 studies 
evaluated team behaviours, processes or emergent 
states (eg, teamwork climate, safety climate, trust); and 
13 studies evaluated clinical processes or patient out- 
comes (see online supplementary appendix). 

Overall, learner reactions to team-training have 
been shown to be positive, often with 80% or more 
of training participants, indicating that the topics 
covered during training were applicable to their work 
setting (see online supplementary appendix). 
Reactions can be important factors in shaping learner 
motivation to use effective teamwork strategies on the 
job and should be considered for that reason; 
however, they are not highly correlated with other 
training outcomes 64 and are not generally considered 
strong indicators of training effectiveness. Increasingly 
evaluation efforts have moved away from reporting 
participant reactions as primary outcomes, focusing 
instead on evaluating effects on provider behaviours 
and patient outcomes. 

Level 2: learner knowledge, self-efficacy and attitudes 
This second level of evaluation examines the impact 
of training on learner knowledge, self-efficacy and 
attitudes. Previous reviews indicate that such out- 
comes were reported in 50% 13 to 78% 27 of studies. 
Self-efficacy or confidence in one's ability to engage 
in effective teamwork behaviours during future clin- 
ical events was the most commonly reported outcome 
at this level among the studies reviewed in table 2. 
Self-efficacy is an important antecedent that shapes 
intentions to use effective teamwork skills during clin- 
ical practice and has been related to increased use of 
effective teamwork behaviours. 65 Overall, previous 
reviews and the additional seven primary articles that 
evaluated these outcomes provide evidence that team- 
training can positively impact these types of outcomes 
(see online supplementary appendix). Of the seven 
articles, five reported significant improvements in 
teamwork attitudes, self-efficacy or knowledge follow- 
ing training and the majority of these used CRM or 
TeamSTEPPS training curriculums. Several studies 
have also investigated the duration of these effects 
over time or differential effects among various types 
of clinicians. For example, Figueroa 65 found that 
increases in leadership and clinical skills confidence 
achieved immediately post-training were sustained at a 
3 -month post-training evaluation. Similarly 
Patterson 66 found gains in teamwork knowledge sus- 
tained at follow-up conducted 8-10 months 
post-training. 

Level 3: teamwork behaviours, processes and emergent states 
Changes in teamwork behaviours and processes are 
arguably the primary criteria by which the effective- 
ness of team-training programmes in healthcare has 
been evaluated. Previous reviews found that 44% 28 to 
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63% 14 of studies reported evaluating the impact of 
team -training on team behaviour or performance. 
Sixteen of the twenty-six studies (62%) reviewed (see 
online supplementary appendix) evaluated the impact 
of team-training at this level, of which 12 reported 
significant improvements (ie, statistically significant 
improvement on one or more process measures or at 
least one dimension of safety culture/climate, or statis- 
tically significant differences between participating 
teams and controls). The methods used to assess team 
performance range from team self-ratings to blinded 
observational ratings using validated measurement 
tools. A number of observational tools and behav- 
ioural marker systems have been developed to evaluate 
teamwork in both simulated and live practice environ- 
ments (eg, the Oxford Non-Technical Skills 
(NOTECHS) scale, 67 the Anesthetists' Non-Technical 
Skills (ANTS) system 68 ), which have greatly improved 
the strength and validity of findings in studies that 
have used them. For example, one high-quality study 
that used a modified version of the NOTECHS 
adapted for the trauma care setting found significant 
improvements in observed teamwork processes both 
during simulated cases and live cases following an in 
situ simulation-based team-training programme that 
included both a brief web-based didactic learning 
session and multidisciplinary clinical simulations. 

In addition to team behaviours, evaluations of team- 
training have also often included measures of emer- 
gent, affective aspects of team performance such as 
teamwork climate or safety climate. Teamwork climate 
and safety climate reflect shared perceptions regarding 
communication openness, mutual support and other 
aspects of team functioning. Previous reviews also indi- 
cate that such measures may be used as part of pretrain- 
ing needs analysis. 27 They are often measured with a 
safety culture or climate survey; however, reporting of 
results is highly heterogeneous. Some studies report 
composite scores that aggregate several survey items 
into one domain or dimension (eg, Carney, 69 Phipps 70 ) 
while others report item-level results (eg, Stevens 71 ). 
Overall, there is some moderate evidence that team- 
training may positively impact teamwork climate or 
safety climate outcomes; however, conclusions are 
limited by variation in reporting. 

Level 4: clinical processes and patient outcomes 
More recently, studies demonstrate the impact of 
team-training programmes on clinical processes, as 
well as indices of care safety and quality. Previous 
reviews found that only a few studies had attempted 
to evaluate the impact of team-training on such out- 
comes. 5 13 14 27 72 However, effect sizes have tended 
to fall in the medium to large range (eg, d=0.63, 73 as 
cited in 5 ), indicating that team-training has been 
related to meaningful improvements in clinical process 
and patient outcomes. Thirteen of the twenty-six 
studies summarised in table 2 evaluated the impact of 



team-training at this level, of which ten reported stat- 
istically significant improvement. Additionally, several 
robust studies offer high-quality evidence that com- 
prehensive team-training programmes can improve 
both clinical processes and patient outcomes. For 
example, an evaluation of the VA MTT programme 
demonstrated significant and sustained decreases in 
preoperative delays (from 16% to 7% of cases, 
p=0.004), increased antibiotic prophylaxis compli- 
ance (from 85% to 97%, p<0.0001), decreases in 
equipment issues/case delays (from 24% to 7% of 
cases, p<0.0001), decreased handoff issues (from 
5.4% to 0.3% of cases, p<0.0001) and most notably 
significantly greater reductions in both mortality 
(RR=1.49) and morbidity (RR=1.20) for facilities 
that implemented team-training. 38 58 Furthermore, a 
dose-response relationship was established such that 
for each quarter the programme was in place at a facil- 
ity, a decrease of 0.5 deaths per 1000 procedures 
(p=0.001) was observed. Implementation of 
TeamSTEPPS has been associated with increased effi- 
ciency in clinical processes for multidisciplinary 
trauma teams (eg, decreased times from arrival to 
surgery from 130.1 to 94.5 min (p<0.05), endo- 
tracheal intubation from 10.1 to 6.6 min (n.s.) and 
CTscan from 26.4 to 22.1 min (p<0.01), 74 as well as 
an 83% reduction in medication and transfusion 
errors (p<0.001) and a 70% reduction in needlestick 
injuries and exposures (p<0.05) in a U.S. Combat 
Support Hospital deployed in Iraq. 39 Other studies 
have also reported significant reductions in clinical 
decision time (p<0.05) 40 associated with team- 
training, as well as one study showing a reduction 
in adverse clinical events and a 50% reduction in 
high-severity malpractice claims (pretraining 11 high- 
severity claims, post-training 5 high-severity claims, 
no statistics reported). 75 

Strength of evidence 

Overall, the body of evidence concerning the effect- 
iveness of team-training in healthcare spans the con- 
tinuum of low-quality, medium-quality and 
high-quality studies. The majority of previous reviews 
did not systematically rate study quality. However, 
Buljac-Samardzic 13 concluded that the majority of 
studies reviewed were of low-level to moderate-level 
quality even though eight of the reviewed team- 
training studies were categorised as high or moderate 
quality (ie, RCT or high-quality pre-post study). 
Similar conclusions were drawn by McCulloch. 28 
Significant homogeneity among studies, both in train- 
ing content and strategy, as well as reported outcomes 
limited conclusions. In the review by Rabol 27 of 
classroom-based team-training interventions, 15 of the 
18 reviewed studies were uncontrolled and 17 studies 
were rated at a moderate or high risk for bias. 

A number of high-quality primary studies have been 
published since these reviews were completed. These 
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include the \A studies 38 58 69 and several studies of 
the TeamSTEPPS curriculum. 35 39 42 These studies 
used high-quality retrospective controlled designs or 
high-quality pre-post designs that examined the effects 
of training across multiple outcomes including team- 
work processes and patient outcomes. Several very 
recent studies published after our search window 
ended have also employed strong quasi-experimental 
designs, larger sample sizes, robust analytical methods 
(eg, multilevel analyses) and different outcomes (eg, 
care coordination practices across teams, communica- 
tion with patients and families) to further strengthen 
conclusions regarding team-training effectiveness. 76-79 

Are the effects of team-training sustained over time? 
Sustainability of team-training effects over time is a 
common question among both improvement scientists 
and practitioners. Many early evaluations of team- 
training limited follow-up evaluations to 6 months 
post-training or less. 27 28 Recent studies have included 
longer follow-up evaluation periods with multiple 
measurement points. Of the 26 papers summarised in 
table 2, 11 included follow-up evaluations examining 
at least one of the Kirkpatrick levels more than 
6 months after the training activity. For example, 
Mayer and colleagues 35 found significant improve- 
ments in observed team processes 1 month post- 
training that were sustained at 6 months and 12 
months post-training in a sample of critical care provi- 
ders. Findings regarding changes in survey measures 
of patient safety culture over time were mixed, 
however. In studies examining the implementation of 
the \A's team-training programme, results indicated 
both significant improvements in perceptions of team- 
work climate 9-11 months following implementa- 
tion 69 and statistically significant reductions in 
surgical mortality 1 year post. 38 58 Several studies also 
demonstrated that variability in aggregate indices of 
patient harm were reduced following implementation 
of a team-training programme delivered using a com- 
bination of both didactic learning activities and in situ 
simulation. 36 70 Overall, there is evidence for 
improvements in teamwork knowledge, attitudes and 
team processes during shorter term evaluation 
windows (eg, less than 6 months) and increasingly 
more robust evidence for longer term impact on 
aggregate indices of real or potential patient harm. 
There are also indications of skill decay over time, 
however, with several studies demonstrating that all 
short-term improvements in knowledge, skills and 
practices were not maintained over longer time 
periods. 66 80 While a great deal of work has been 
dedicated to examining optimal refresher training 
intervals for clinical procedural skills (eg, resuscitation 
or intubation skills 81 82 ), relatively minimal work has 
been done to understand optimal refresher intervals 
for teamwork competencies. 



DISCUSSION 

In summary, previous reviews of team-training in 
healthcare and recent studies have found that these 
programmes can improve teamwork processes and 
patient outcomes (eg, reductions in adverse events, 
reductions in mortality and morbidity). Current evi- 
dence also offers detailed insight into team-training 
design and implementation strategies relevant for 
researchers and practitioners alike. 13 14 24 27-32 
Additionally, over three decades of research examining 
team-training in a wide variety of contexts exists that 
both researchers and practitioners can draw from. 

In terms of the strength of evidence, previous sys- 
tematic reviews included several studies that used RCT 
or controlled pre-post designs. Several robust studies 
examining team-training strategies have been published 
since previous reviews concluded. However, it is 
important to note that previous reviews and recent 
studies reflect a wide range in quality of evidence — 
with several studies limited by small sample sizes, weak 
study design and limited detail regarding the team- 
training curriculum or implementation strategy. 
Overall, our synthesis suggests that there is moderate 
to high-quality evidence that team-training can posi- 
tively impact healthcare team processes and, in turn, 
clinical processes and patient outcomes. These effects 
have been demonstrated across a variety of acute care 
areas, including critical care, surgery and emergency 
care. A key finding is that the studies demonstrating the 
most robust evidence for effectiveness 35 38 58 have 
implemented team-training as a bundled intervention 
that includes preplanning, readiness assessment, inter- 
disciplinary learning activities and tools to support 
active transfer and sustained use of effective teamwork 
practices into daily care. Additionally, evaluation 
efforts undertaken in the most robust studies leveraged 
large care systems and strong quasi-experimental evalu- 
ation designs. This is not to say that every attempt to 
implement team-training should be compelled to 
follow similar evaluation efforts. These findings do 
underscore the value of implementing bundled inter- 
ventions that match learning activities with activities to 
actively practice effective teamwork strategies and 
tools to support effective teamwork in daily practice. 
This underscores that team-training is most effective 
when paired with other methods to improve teamwork 
and also that team-training is one of several interven- 
tions that has proven effective at reducing patient harm 
and improving patient safety. 62 83 

Additionally, our synthesis underscores that there is 
meaningful heterogeneity and local adaptation in both 
training design and implementation across studies. 
While the studies reviewed used similar general training 
strategies (eg, CRM), the details of specific competency 
areas targeted and delivery methods were highly hetero- 
geneous. As noted in previous reviews 13 14 28 and in the 
more recent work reviewed here, training programmes 
varied in duration, activities and modality. 
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The importance of local customisation is also reflected 
in the toolkits available to support team-training imple- 
mentation. For example, a customisable version of the 
TeamSTEPPS curriculum is available publically through 
AHRQ (http://teamstepps.ahrq.gov/). Additionally, the 
VA MTT programme is available to VA Medical 
Centers through the National Center for Patient Safety 
(http://www.patientsafety.gov/mtt/). Both programmes 
provide customisable content while emphasising local 
adaptation of learning materials and implementation 
strategies. Best practices for team-training development 
and evaluation 61 84 can also provide guiderails for 
implementation and evaluation design. 

To continue building this evidence base, future work 
should continue to evaluate team-training efforts 
across time and outcomes. This includes continuing to 
examine the impact on patient safety outcomes using 
robust implementation designs, evaluating team- 
training effects in multiteam system contexts (eg, 
chronic disease management, comprehensive cancer 
care), examining the comparative effectiveness of dif- 
ferent methods of training implementation and exam- 
ining implementation methods to support sustainment 
of behaviour changes achieved through training. For 
example, there is currently limited evidence that pro- 
vides insight into the frequency of retraining or dedi- 
cated practice needed to develop and maintain 
effective teamwork skills. Additionally, there is a need 
to examine how dynamic team composition (ie, 
changes in team membership) moderate team processes 
and the effects of team-training. Methodologically, 
robust validation studies can continue to strengthen 
the evidence regarding team-training effectiveness and 
the factors or conditions that influence effectiveness. 
Finally, longitudinal studies and studies addressing the 
integration of team-training concepts throughout the 
career development of healthcare professionals are 
needed to continue building our understanding of 
team-training in healthcare. 
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